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Abstract. Motivated by the psychological literature on the “peak-end rule” for 
remembered experience, we perform an analysis within a random walk framework of a 
discrete choice model where agents’ future choices depend on the peak memory of their 
past experiences. In particular, we use this approach to investigate whether increased 
noise/disruption always leads to more switching between decisions. Here extreme value 
theory illuminates different classes of dynamics indicating that the long-time behaviour 
is dependent on the scale used for reflection; this could have implications, for example, 
in questionnaire design. 


1. Introduction 

The use of stochastic processes in interdisciplinary modelling has a long history 
dating back at least to Bachelier’s seminal work in finance [1] and encompassing 
applications to traffic fiow [2], biological processes [3] and opinion dynamics [1], 
among others. Often such systems are analysed using a Markovian (or memoryless) 
approximation which considerably simplifies the theoretical treatment. However, within 
the statistical mechanics community there is much topical interest in characterizing the 
properties of non-Markovian models. There are many ways to incorporate memory 
effects including generalized Langevin or Fokker-Planck approaches num, and the 
assumption of internal variables or non-exponential waiting times in many-particle 
microscopic models [U |9l [TOl. At the random walk level, recent analytical studies in 
the physics literature have included the imaginatively named “elephant” random walker 
who remembers a property of the entire history m, the “Alzheimer” random walker 
who recalls just the distant past [I21[I3], and “bold” and “timorous” random walkers 
who behave differently only when they are at the furthest point ever attained [H]. In 
fact, the elephant random walk can also be related to the older Polya urn problem [T5] : 
see |TB] for a mathematical review of this and other random processes with reinforcement. 
Amongst the more recent rigorous results are some for the “excited” (or “cookie”) 
random walk [muH], random walks with different kinds of self-interaction [T^ 120] , and 
those with internal states 1211 . 
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In real-life social and economic scenarios the dependence on memory is, of course, 
rather complex. However, one psychological heuristic is the “peak-end rule” suggested 
by Kahneman et ah izg. This asserts that the remembered utility (loosely speaking 
the pleasure or pain experienced) of a specihc situation/episode is approximately given 
by the average of the peak experience (best or worst) during the event and the hnal 
experience of that event. Notice in particular that this implies “duration neglect” |23] 
in the sense that the extreme and hnal snapshots are considerably more important in 
the memory than the overall length of the experience (even if it is an unpleasant one!). 
Empirical support for this peak-end approximation comes from situations ranging from 
the pain of medical procedures [21] to the pleasure of material goods [21] . Whilst other 
work paints a more complicated picture, particularly for extended events [26], it is 
clear that peak experiences play an important role and, to the best of our knowledge, 
such memory of extreme values is largely unexplored from the perspective of statistical 
physicsIU 

In this spirit, our contribution is to consider a random walk model where the 
probability of moving left or right depends on the maximum value of a random variable 
associated to each time step. As we will show, this can be thought of as a simple discrete 
choice model with a dependence on the “peak” of past experience. In particular, we 
use this framework to investigate whether increased noise in the model (corresponding 
perhaps to the “churn” of changing circumstances or some kind of disruption, cf. 
e.g., [27]), always leads to more switching between decisions. Using the mathematics of 
extreme values, we show that the answer to this question depends on the distribution of 
the random variable encoding the experience at each step. Our work thus helps to shed 
light on real-world issues as well as contributing to building up general understanding 
of memory effects in statistical mechanics models. 

The remainder of the paper is structured as follows. In section we describe our 
random walk formalism and explain its signihcance as an opinion choice model as well 
as the manner in which it extends previous work on generalized Polya urns. In section]^ 
we employ extreme value theory to develop a heuristic argument for different classes 
of long-time behaviour depending on the distribution of past experience, and compare 
our predictions with simulations. Finally, in section]^ we conclude with a discussion of 
implications and open questions. 

2. Random walk set-up and interpretation as decision model 

We consider a one-dimensional random walker who steps right or left in discrete time, 
denoting by the number of steps right up to time t and XR the corresponding 
number left. Note that XR = t — Xi^ by construction. For later convenience, we also 
dehne the corresponding time averages (“velocities”) V~^ = X~^/t and V~ = X~/t, 
suppressing the notational dependence on t where no confusion should arise. 

f Note that this is subtlely different from problems in which the dynamics depends on prediction of 
future extreme values as relevant, for example, in financial applications. 
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In addition, at each time step i we associate an independent identically distributed 
(i.i.d.) random variable Ui from some known distribution with cumulative distribution 
function (c.d.f.) F{u). Crucially, the walker “remembers” the maximum value of Ui 
for all rightward steps in its history and, separately, the maximum value of Ui for all 
leftward steps. We denote these history-dependent random variables by U^ and Uf 
respectively so that formally we have 


Uf = max|t/j 

i<i<t ^ 


Xt - -V-i = !}■ 


( 1 ) 


Memory is then built into the dynamics via the setting of left and right hopping 
probabilities for the next step to depend on the current values of U^ and U~. It is clear 
that the system is non-Markovian in position space although, of course, still Markovian 
in an enlarged state space including f/+ and U~. 

The central idea is that this set-up is analogous to a single agent in a discrete 
decision model where U is some kind of “utility” and the agent remembers its extreme 
value (corresponding to the “peak” part of Kahneman’s peak-end rule) for each of two 
choices. Specihcally, we £x the right and left stepping probabilities as functions of the 
random variables U~^ and U~ to accord with the familiar “logit” choices of economic 
theory 

f,U+/T pU-/T 

p+ = _ p- = _ (2) 

qU+/T qU-/T ^ qU+/T _i_ qU-/T t 

where the positive parameter T represents the level of noise in the decision|§] Throughout 
the paper we set Uf^ = Uq = 0 so that the two choices (step directions) are initially 
equally likely; as the system evolves the jump probabilities become asymmetric due to 
differing values of U~^, and U~. In particular, note that U~^ and U~ are monotonically 
increasing with the number of steps right and left respectively. 

In passing, we note here that if U^ were deterministic functions of the velocities 
the model would closely resemble the Polya urn problem, familiar in the mathematics 
literature [I5] , where the probability of selecting a ball of a particular colour depends on 
the fraction of that colour chosen previously (in a similar manner, the elephant random 
walker of m steps left and right with probabilities depending on the relative number of 
such steps in his past). If, as here, the probability function is nonlinear, the urn model 
is known as a generalized Polya process [iniEgiEn]. The crucial difference in our model 
is that 17^ fluctuate in a correlated way due to the statistics of the extreme values - 
we seek to determine the effect this has on long-time properties such as the average 
velocity of the random walker (or, equivalently, the proportion of time the agent makes 
each decision). 

One might naively expect that, as for the Polya models, in the large t limit our 
random walker approaches a hxed-point state where the relative probabilities (and hence 
the fraction of steps left and right) do not change. The symmetry of ([^ suggests two 

§ The symbol T is chosen here to reinforce the analogy with temperature in a physical system and is 
not to be confused with a time parameter; the update probabilities (H loosely resemble the Glauber 
refreshment formulae for an Ising model [55]. 
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specific types of fixed point; (i) (i4+, V~) = (1/2,1/2) with and U~ asymptotically 
eqnal and hence symmetric behavionr of the random walker, i.e., both choices eqnally 
likely in the long run; (ii) = (0,1) or = (1,0) with one of U'^ or 

U~ negligibly small with respect to the other and hence an asymmetric random walker 
moving only in one direction, i.e., the agent frozen in one or other choice. 

We shall demonstrate the existence of these hxed points more carefully later. For 
now, we remark that a pertinent question relates to their stability, in particular, whether 
the symmetric hxed point (1/2,1/2) can be made stable by increasing the noise. This 
could be important for sharing the load between two different choices (e.g., two different 
routes or transport options). In the next section, we address this issue for different 
distributions of U before, in section]^ considering the added effect of the “end” part of 
the peak-end rule. 


3. Extreme value controlled behaviour 


3.1. Outline of method 


The behaviour of the model will obviously depend on the distribution of U. Our strategy 
is hrst to analyse the typical long-time dynamics by approximating in ([^ by the 
so-called “characteristic largest value” of extreme value theory and then, where relevant, 
to consider the added effect of huctuations about this. 

The characteristic largest value after trials is dehned for a given F{u) as the 
value of u at which F{u) = 1 — 1/X^. It gives a straightforward way to obtain the 
scaling of the maximum value and is closely related to other properties of the full 
distribution [31], as we shall see for various cases in the following subsections. Our 
approach using the characteristic largest value leads to hopping probabilities depending 
on the number of left/right steps over the whole previous history and is thus in the spirit 
of the generalized Polya urn models mentioned above or continuous-time analogues 
with current-dependent hopping rates [32]. One subtlety here is that the resulting 
probabilities in our model depend directly on the number of steps left and right, X^, 
not the fractions, V^. Depending on the functional form of the characteristic largest 
value this may introduce an explicit time dependence in the dynamics for as we 
shall see in some of the subsequent examples. 

In fact, since V~^ -|- V~ = 1 by construction, this procedure enables us for a given 
utility distribution F{u) to write simply in terms oi V = V~^ — V~ and possibly 
time t. Now, if the random variable V takes value v, the mean distance moved in the 
next step is given by the corresponding value of P~^ — P~ which we denote by At(n). 
Hence, on average, we expect a “typical trajectory” given by the discrete mapping 


Vt+l 


tvt + Xtiyt) 
t + 1 


(3) 


For cases where Xt{v) has no dependence on t it is immediately clear from (|^ that hxed 
points V* should satisfy v* = A{v*) and a standard “cobweb”-type construction predicts 
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Figure 1. Sketch graphs showing iteration of Vt using in vicinity of stable fixed 
point (left) and unstable fixed point (right). The slope of Vt+i as a function of Vt is 
(t + A'{vt))/{t + 1) which approaches unity as t increases (indicated by dashed lines) 
leading to slow decay/growth. 


that the stability of the hxed points is determined by the slope of the increasing function 
A(n), see £gure[^ Specihcally, if A'(n*) < 1 [£gure[^a)] then small fluctuations below 
V* are characterized by A(v) > v, so on average the velocity increases back towards 
the hxed point. Similarly, huctuations above v* have A(r;) < v, so on average the 
velocity decreases, again back towards the hxed point. An analogous argument shows 
that hxed points with A'{v*) > 1 are unstable [hgurej^b)]. Notice that due to the time 
dependence of the mapping (|^ the decay towards hxed points is expected to be power- 
law rather than exponential in nature - physically this is because as the measurement 
time increases the last step has a smaller and smaller ehect on the overall time average. 

In the following subsections we illustrate this approach for three qualitatively 
diherent scenarios corresponding to the three known families of extreme value theory. 
(In cases where Aj(u) itself depends on time we shall chiehy be interested in its behaviour 
as t —)■ cxD.) We then confront the predictions with simulation results and discuss how 
huctuations in the extreme values modify the picture of typical behaviour given above. 


3.2. Exponential tails 


To demonstrate the method, we hrst look in detail at the case where the utility variable 
U has an exponential distribution with c.d.f. 

F{u) = 1 - e-^“, M > 0. (4) 


Here the characteristic largest value after steps is given by (In X=*") /A so, substituting 
for in (j^, we approximate in the long-time limit by 

(X±)V(AT) 


= 


(X + )1/(AT) (X-)V(AT) 


( 5 ) 
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or equivalently, in terms of the time averages, 

P* =__ (6) 

("V^+^l/CAT) 

In this particular case, the probabilities can be written in terms of without explicit 
time dependence illustrating a direct connection to the class of elephant random walker 
and (time-homogeneous) Polya urn problems. 

To determine the fixed points we further write in terms of the net velocity by 
substituting = (1 ± V^)/2 to obtain 


(1 + y)i/(Ar) _ (1 _ 

(l + ^)l/(AT) + (l_^)l/(AT)’ 


(7) 


so the function specifying the mean displacement of the next step can be compactly 
written as 


A(n) 


tanh 



1 

1 — V ) 


( 8 ) 


where v is the current value of the velocity. The fixed points satisfying v* = A(n*) are 
then seen by inspection to be v* = 0, ±1 as predicted from symmetry arguments. 

Recall from the previous subsection that to determine which of these fixed points 
is stable we need to check the slope A'(n*); here it is straightforward to show from (|^ 
that 

A'(0) = (9) 

Hence if AT > 1, the mixed solution (R^, V~) = (1/2,1/2) is stable and the asymmetric 
frozen solutions (R+,R“) = (1,0) and (R+,R“) = (0,1) correspondingly unstable. 
Similarly, for AT < 1, the mixed solution is unstable and we predict that the random 
walker becomes frozen into ballistic motion in one of the two directions. To check this 
heuristic argument we appeal to Monte Carlo simulations - in figure we show the 
empirical distribution of velocities at f = 100 for an exponential utility distribution 
(with mean A = 1) and values of noise predicted to correspond to the two different cases 
(T = 0.8 and T = 4.0). 

We see good qualitative agreement of the simulations with the prediction: in the 
low-noise case the trajectories are sharply peaked around the asymmetric fixed points, 
i.e., V = V~^ — V~ = ±1 (corresponding to each agent almost always making the same 
choice) whilst in the high-noise case the trajectories are clustered around the symmetric 
fixed point, i.e., V = V~^ — V~ = 0 (corresponding to each agent sampling the two 
choices approximately equally). However, there is a hnite width of the distribution 
about the fixed point (s) even for AT significantly greater than unity ~ to investigate 
this more systematically, and reveal possible finite-time effects, we plot in figure the 
standard deviation of the distribution as a function of T for increasing measurement 
times. This quantifies how close the trajectories end to symmetric or asymmetric fixed 
points without making a distinction between the two asymmetric states (whose selection 
is expected to depend sensitively on the agent’s first few choices). According to the 
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V 


Figure 2. Histogram of final velocity V = — V~ for random walkers with 

dependence on peak values of exponential utility distribution (A = 1) and two different 
noise levels (T = 0.8 and T = 4.0). Distribution calculated from 10® trajectories each 
running up to hnal time t = 100. 



T 


Figure 3. Standard deviation of final velocity V = F+ — V~ for random walkers 
with dependence on peak values of exponential utility distribution (A = 1) and range 
of noise values. Points show simulation results for increasing times (10® trajectories in 
each case); solid line is numerical solution of (12); dashed line is approximation (B.8|. 
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analysis of typical behaviour given above, we expect the standard deviation to be unity 
for AT < 1 and zero for AT > 1. In fact, although the simulations do show evidence of 
a transition around AT = 1, the situation is somewhat more complicated; in particular, 
the standard deviation clearly converges to a finite value even for AT > 1. 

These observed results for the standard deviation ay suggest that, even in the long¬ 
time limit, the properties of the model are sensitive to the full distribution of maximum 
values not just the characteristic largest value. As further evidence of this, we remark 
that if were given deterministically by the characteristic largest value, the variance 
for AT > 1 could only be due to decay towards the stable fixed point and fluctuation of 
individual trajectories about the typical behaviour. In this case, V would be expected 
to obey a large deviation principle with some “speed” [321 133] and the variance 
would eventually converge to zero, as confirmed in Appendix A where, for comparative 
purposes, we present simulation results from an artificial model with at every time 
step set equal to (In A^)/A. It is clear then that the limiting value of the variance in the 
full model is determined by fluctuations in the extreme values, leading to fluctuations 
of the typical trajectories themselves. 

In the case of an exponential distribution it is, of course, well known that the 
limiting form of the rescaled maximum has a Gumbel distribution; here the c.d.f. of 
is asymptotically given by 

G(n±) = (10) 

where = lnX=*"/A and 6^ = 1/A (see, e.g., [211 El] and references therein). The mode 
coincides with the characteristic largest value calculated earlier while the mean is 
-I- 6^7 (with 7 the Euler-Mascheroni constant) so differs from it only by a constant 
amount. Taking account of the fluctuations, the maximum value random variables thus 
obey 

lnX+ hiA- 


u+ -u- = 


+ e 


( 11 ) 


A A 

where the distribution of e is given by the difference of the two Gumbel distributions as 
a logistic distribution with mean zero and variance cr/ = 7r^/(3A^). 

Substituting the form of ( pT| in the expression for — P~ and repeating the 
calculations leading to ([^ one finds that for a given, non-zero, value of e the position 
of the “symmetric” fixed point is shifted from zero although both its stability and the 
position of the asymmetric fixed points remain unchanged. A crude estimate of the 
standard deviation in the position of the symmetric fixed point can be obtained as the 
value of n G (—1,1) which solves the transcendental equation 


V = tanh 


In 


1 -I- n 


2T 


( 12 ) 


2\T “U 

As seen in figure [^this leads to a loose upper bound on the observed standard deviation. 
The actual standard deviation is smaller because the value of e and the corresponding 
fixed point changes during the course of each trajectory. In Appendix B we include this 
effect within a linear expansion to obtain an analytical expression for ay which is a better 
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Figure 4. Standard deviation of velocity V = V~^ — V~ against time for random 
walkers with dependence on peak values of exponential utility distribution (A = 1) and 
change in noise level at t = 500: T = 0.3 to T = 3.0 in red (+ symbols), T = 3.0 to 
T = 0.3 in green (x symbols). Inset shows final {t = 10000) velocity histograms in the 
two cases. All calculations from 10® trajectories. 


approximation for AT large (see, again, figure]^. Notwithstanding the finite variance, 
the claim that one can control the long-time behaviour by increasing/decreasing the 
noise is well borne out by simulation. For example, in figure we show the evolution 
of the standard deviation in a scenario where the noise level (and hence the stability of 
the fixed points) is abruptly changed after the first 500 time steps. 

The cornerstone of extreme value theory, the Fisher-Tippett-Gnedenko theorem |35l 
I3B] . asserts that the Gumbel distribution is universal for the rescaled maximum of 
i.i.d. random variables drawn from a distribution with exponential tails. However, the 
functional form of the scaling parameters depends on the distribution being considered. 
As a second example, we now make the arguably reasonable hypothesis that agents 
assign utilities according to a Gaussian with some mean fi and standard deviation 
The mode of the limiting distribution, again given by the characteristic largest value, is 

= ia-g^-\l/X^) (13) 


where <F is the c.d.f. of the standard normal distribution. We note that, in this case, 
retains a logarithmic dependence on growing like gy/2 lnX=*= as —)• oo. Ignoring 

the fluctuations about this value, an analogous argument to that given above then yields 
for large t 


g 

2T 


2 In 


(1 + v)t 
2 



Ib^D) 


Xtiy) = tanh 


(14) 
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T 


Figure 5. Same as figure but for Gaussian utility distribution (y = 1, <^ = 1). 


It is clear that v* = 0 is a fixed point for all t and its stability is controlled by the 
slope 


ry21n(«/2)' 


(15) 


As t —)■ cxD the slope tends to zero and hence we predict that the symmetric fixed point 
is always stable in the long run. However, since the dependence is only logarithmic 
in t, one still expects to see a noise-controlled transition for large but finite times. This 
is supported by the simulation results for standard deviation shown in figure For 
comparison, we have set there the first two moments equal to those of the exponential 
distribution in figure and the picture for the Gaussian case is qualitatively similar 
with the transition between low and high-noise regimes only weakly dependent on t. To 
complete the story, we can again consider fluctuations of the maximum values. In this 
case, the width of the Gumbel distribution is controlled by 

= -- 

$-1 ( 1 / X ±) 

which decays to zero as —)■ oo (again see, e.g., [211 Ell)- Hence, in contrast to the 

exponential case, we do not expect a finite limiting velocity variance in the high-noise 
regime and indeed the relevant simulation data do seem to show a slow convergence 
towards zero. 

A similar argument applies to other distributions with exponential tails - the 
characteristic largest value of converges to the mode of the corresponding Gumbel 
distribution and generically grows as (InX^)'' where the power k determines the long- 
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Figure 6. Same as figure|^but for Gaussian utility distribution {p, = 1, q = 1). Note 
that in the low-to-high noise case with these parameters, two peaks are still visible in 
the velocity histogram at t = 10000; they are a residual effect from the first 500 time 
steps and merge if the final time is further increased. 


time stability of the symmetric fixed point via 


a;(o) 


T 


(17) 


The exponential distribution (|^ corresponds to the special case oi k = 1, while for k < 1 
we expect long-term stability of the symmetric fixed point (A((0) < 1) and, for k > 1 
we expect long-term instability (A((0) > 1). For intermediate timescales, the system 
can be driven towards either the symmetric mixed state or the asymmetric frozen state 
by increasing or decreasing the noise, as we demonstrate for the Gaussian distribution 
in hgure 


3.3. Power-law tails 

The second class of extreme value statistics corresponds to distributions with power-law 
tails as typihed by the Pareto distribution with c.d.f. 

F{u) = 1 - , u>Um (18) 

where Um is a lower bound and a > 0. In this case one finds that the characteristic 
largest value after steps is given by Um{X^Y^°‘ leading to the approximation 

= 




( 19 ) 
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T 

Figure 7. Same as figure]^ but for Pareto utility distribution (xm = 0.5, a = 2). 


and hence, by the same method as previously, 


At(n) = tanh 


u, 


j-l/o 

7^ 


2T 


1 + n 


1/a. 



( 20 ) 


Again, for all t we hnd a symmetric hxed point at v* = 0, here with stability determined 
by the slope 

( 21 ) 

which is greater than unity for t > 2{Ta/um,)°‘- In fact, in the limit t — )■ oo, At{v) 
approaches the step function sgn(n) with corresponding stable hxed points at v* = ±1. 

In this Pareto case, it is straightforward to show that for large the maximum 
value has approximately a Frechet c.d.f. 

G{u^) = > 0 ( 22 ) 

where the scale parameter is given by The mean of this distribution 

is only hnite for a > 1 but the mode and the median are both proportional to so, 
again, the trivially calculated characteristic largest value should give a good indication 
of the long-time behaviour. This is conhrmed in hgure where the standard deviation 
of the velocity against noise strength is plotted for a case where the utility has a Pareto 
distribution with unit mean {xm = 0.5, a = 2). For all values of T, the velocity 
variance converges towards unity (corresponding to individual trajectories approaching 
the asymmetric hxed points at v* = ±1). We have also checked that the convergence is 
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faster for smaller values of a (“longer tails”), noting in particular that the distribution 
of U has inhnite mean for a < 1. 

More generally, the Frechet distribution is the limiting form for the rescaled 
maximum of i.i.d. random variables drawn from any distribution with power-law 
tails [2S1IMI- hi all such cases we expect that A((0) increases as some power of f, leading 
each agent to ultimately become frozen in a pure state corresponding to one or other 
choice. We remark that this power-law dependence is stronger than the logarithmic 
form found in section 3.2; even by increasing the noise we only expect to be able to 


favour the mixed state for short timescales, e.g., up to the order of (Ta/um)°' for the 
Pareto distribution considered above. 


3.4- Bounded distributions 

Finally, we consider distributions of U with hnite upper bound (as might be appropriate, 
for instance, if an agent’s memory is based on some predetermined numerical scale with 
given minimum and maximum). The obvious example is a uniform distribution with 

c.d.f. 


F(u) = 


iij tie [;,!■) 


(23) 


1 u > r 

whose characteristic largest value after steps is given by r — {r — l)/X^. Notice that, 
in contrast to the previous examples, this converges to a hnite constant as X^ —)■ cx) 
which is an elementary consequence of the upper bound on the underlying distribution 
and already gives a hint at the long-time behaviour. 

In this case, following our previous heuristic procedure we have 

p-(r-/)/(TV±p 

(24) 


= 


g-(r-0/(TV+t) ^ g-(r-0/(TV-t) 


and 


At(u) = tanh 


/ 


Tt (1 — u 1-l-u 
The slope at the symmetric hxed point is given by 

2(r — 1) 


A'(0) = 


Tt 


(26) 


(26) 


which is less than unity for t > 2{r — l)/T and tends to zero as t —)■ cx). Hence we 
argue that the symmetric hxed point is always stable for long enough times (regardless 
of noise strength). This conclusion is supported by the simulation data in hgure 
The observed behaviour of the variance for very small T can be roughly explained by 
noting that, for this version of the model, the walker can become stuck for hnite times 
in a metastable hxed point at v* = ±ljj]] To see this, we plot in hgure the function 


Xt{v) of (25) and examine its intersections with the line u, for hxed T and increasing 
t. Notice that, in this case, for t > 2(r — l)/T both symmetric and asymmetric hxed 


II In fact, initial effects may also play a complicating role here since the characteristic largest value 
argument is strictly only a good approximation when the number of steps in both directions is large. 









^t(^) 
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T 

Figure 8. Same as figure]^ but for uniform utility distribution {I = 0, r = 2). 



V 


Figure 9. Plot of At(u) given by (25) for low noise and increasing times. Note 
intersections with diagonal v (solid red line) and compare sketch graphs in figure 
Parameters are I — 0, r = 2, and T = 0.1. 
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points are stable but separated by an unstable point whose position tends to ±1 as 
t ^ oo. The corresponding potential landscape has metastable states at n* = ±1 and 
a trajectory can be trapped in such a state until fluctuations drive it over the barrier 
(whose relative height decreases with time) to the global minimum at v* = 0. We 
emphasize that, since the flxed point at n* = 0 is always stable except for very short 
times, the long-time behaviour of the system cannot be effectively controlled by altering 
the noise (confirmed by further simulations, not shown). 

It is easy to show that, for large X^, the maximum of i.i.d. uniform random variables 
has approximately the reversed (unit) Weibull distribution 

G{u^) = u±<r (27) 

with scale parameter = (r — l)/X^ and mean coinciding with the characteristic 
largest value calculated above. However, once again, the argument is more broadly 
applicable - for bounded distributions the limiting distribution of the rescaled maximum 
is generically reversed Weibull (also known as “Type III” extreme value) with mean and 
median typically approaching the upper bound as some inverse power of the number of 
trials [35l |36]. In all such cases, A((0) —)■ 0 as t —)■ oo, meaning each agent is expected 
to ultimately end up in the mixed state with both choices equally likely. Physically, it is 
clear that in the long-time limit the system approaches a standard memoryless diffusive 
model with P~^ and P~ flxed and equal. 


4. Discussion 


In this paper we have performed a detailed analysis of random walkers with peak memory 
dependence. Commensurate with the original motivation of Kahneman’s peak-end rule, 
we now make some comments on the effect of also including an explicit dependence on 
the final value of the utility U. To be precise, we consider the mean of peak and final 
experience so that the right and left hopping probabilities in ([^ are replaced by 


P+ = 


^(C/++C/+)/(2T) 


,((7++r/+)/(2T) ^(U-+UJ)/{2T) ’ 


P- = 


^{U-+Up/{2T) 


^^{U++UJ)/{2T) ^ ^{U-+UJ)/{2T) 


(28) 


where here Uj {Uj) is the value of U corresponding to the last step right (left). Note 
that the are in general smaller and much less strongly correlated than the IP^ so 
we might expect their effect to cancel out on average in the long-time limit. At the 
same time, the dependence on is here weakened in the sense that the values are now 


divided by 2T rather than T. The simulation results in figure IT, for the case of an 
exponential utility distribution, confirm that this modified model behaves very similarly 
to an increased noise version of the original model with the replacement of T by 2T. 

With the preceding paragraph in mind, we argue that our work on the peak memory 
model also has implications for the peak-end case. Specifically, we have found that the 
effect of noise/disruption in the model is dependent on the properties of the utility 
distribution. Using the characteristic largest value to cast the problem as an effective 
Polya process provides direct information on the long-time dynamics (in particular the 
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Figure 10. Standard deviation of final velocity V = V'^ — V~ for random walkers 
with dependence on peak and end values of exponential utility distribution (A = 1) and 
range of noise values. Red points (+ symbols) show simulation data for the peak-end 
model while green points (x symbols) show comparative results for peak dependence 
only but with noise 2T. Both cases calculated from 10® trajectories and 10® time steps. 


stability of fixed points in the system) but, in order to quantify the observed variance, 
one also needs to consider the distribution of maximum values. The examples we have 
shown, together with general arguments rooted in extreme value theory, reveal three 
qualitatively different classes of behaviour: 

• For utility distributions with heavy tails each random walker (agent) eventually 
becomes frozen in a state corresponding to one or other step direction (choice), 
regardless of the level of noise. 

• For bounded utility distributions each agent samples both choices approximately 
equally in the long-time limit, again regardless of the level of noise. 

• For utility distributions with exponential tails the situation is more subtle ~ for 
an e“''‘“ decay we hnd a transition between frozen and mixed states at AT = 1; in 
other cases there is a weak logarithmic dependence on the time. Furthermore, for 
the special case of e“^“ decay, even in the high-noise regime there is a hnite variance 
around the mixed state which can be attributed to fluctuations in the maximum 
values. 

Signihcantly, this implies that only for exponential-tailed utility distributions can one 
hope to increase the switching between decisions on intermediate/long timescales simply 
by increasing the noise. 
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From a statistical physics point of view, it would be interesting in the exponential 
case to characterize the phase transition and scaling exponents at AT = 1, e.g., by 
calculating the correlation function. This latter is also relevant in the opinion dynamics 
context as it quantihes how sensitive the long-time behaviour is to the hrst step and 
thus the extent to which a particular one of the two asymmetric hxed points might 
be favoured by a small initial perturbation. Preliminary simulations suggest that the 
correlation function in the full model converges to zero for AT > 1 and, for AT ^ 1, 
decreases more strongly with AT than in the artihcial model of Appendix A (presumably 
due to the added fluctuations reducing the effect of the initial conditions). However, 
a more detailed analysis with hnite-time scaling along the lines of [30] is deferred to a 
future publication. 

Other extensions of the work might include considering coupled random walkers 
(modelling collective rather than individual memory) or peak effects in other opinion 
dynamics models, such as contact processes and voter models [37]. The peak-end rule 
itself can also be critiqued (see, e.g., the discussion in [26]) and realistic rehnements 
such as the slow fading of peak memories in the distant past could be incorporated into 
the modelling. However, we believe that our current work represents an important hrst 
step beyond simply averaging over the whole past experience or just recalling the most 
recent history. 

Although our analytical calculations thus far have been carried out in the framework 
of a specihc toy model they highlight more generally the possible role of (experienced 
or remembered) utility distributions in maintaining and controlling behaviour. In 
particular, if agents are encouraged to rehect on their own experiences with a view 
to possibly modifying future choices then the outcome could be subtlely dependent 
on any numerical scale offered for rehection, e.g., whether or not it has a hxed upper 
bound. There is much scope here for future interdisciplinary work linking with current 
understanding in psychology and economics. 
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T 

Figure Al. Same as figure]^ but for simplified model (still with A = 1). 


Appendix A. Artificial model for comparison 


Here we present a brief analysis of a simplified model in which at every time step is 
deterministically given by (lnA^)/A (the characteristic largest value of an exponential 
distribution with trials). In this case, the future hopping rates are completely 
determined by the past velocity so previous work on generalized Polya urn models, etc. 
should be directly applicable. 

The standard deviation against noise in this case is shown in hgure [AT] which is to be 
compared with hgurein the main text. For the artihcial model, the standard deviation 
does appear to converge to unity for AT < 1 and zero for AT > 1 as predicted by the 
analysis of typical trajectories outlined in section 3T In hgure |A^ we examine on log-log 
scale the limiting behaviour for selected points in the high-noise regime and clearly see a 
power-law decay. In fact, the asymptotic behaviour of the variance can be predicted on 
general grounds to depend on the slope A'{v*) at the stable hxed point (cf. [321 EH] for the 
continuous-time case). Specihcally, as found in other models mi ESI HQ], one anticipates 
a dynamical phase transition at A'(n*) = 1/2 with dihusive huctuations (i.e., decay 
of the velocity standard deviation) for A'(n*) < 1/2 and superdihusive behaviour (with 
(jgcay) for A'(n*) > 1/2. Setting A'{v*) = 1/(AT), the resulting predictions 
indeed ht the simulation data very well (with logarithmic corrections expected at the 
dynamical phase transition itself). Note that this provides a quantitative explanation 
for the observed slow convergence close to the transition point (AT = 1) which may also 
be relevant in the full model. 

Another important quantity for generalized Polya processes such as this is the 
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Figure A2. Standard deviation of velocity V = F+ — V~ against time for random 
walkers in simplified model (A = 1) at selected noise values (10"^ trajectories in each 
case). Points are simulation data for (top to bottom): T — 1.25,1.5,1.75,2, 3,6. Black 
solid lines are fits corresponding to power laws with negative exponent min(l/2,1 — 
l/(Ar)); logarithmic corrections are expected at AT = 2. 


correlation function C{t) between the direction of the hrst step and the (t+l)th step [30] . 
The long-time limit of this quantity plays the role of an order parameter and simulation 
results in the present case (not shown) conhrm a continuous phase transition at AT = 1 
with C{t) converging to zero for all AT > 1. 

Appendix B. Limiting variance in exponential case 

To understand the fluctuations of the velocity, one needs to take account of the fact 
that each value of persists for a number of time steps before being replaced by a 
larger one. In this appendix, we pursue such an approach with an exponential utility 
distribution to obtain an approximation for the long-time limit of the standard deviation 
in the high-noise regime. 

We let r be the time at which the last record occurred (i.e., the last change in either 
or U~ up to the current time t) and note that, by dehnition, the value of — P~ is 
unchanged for t < i < t. In the exponential case U~^ — U~ is given by 0 and typical 
trajectories should then obey the stochastic mapping 

TVr + {t + l- t) tanh In 

vt+i = (B.l) 

where r and e are random variables and the second term in the numerator gives the 
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expected displacement since the last record. In the case where t = t and e = 0 for 
all time (i.e., or U~ updated at every step with no fluctuations) we recover the 
deterministic mapping of (|^ with ([^. In slight abuse of notation, we denote a typical 
trajectory by Vt even in the present stochastic case and argue that it is the fluctuations 
of this trajectory which lead to the long-time variance of V. 

For large times we can approximate t-\-l ^ t and Vt+i ~ Vt (time-averaged velocity 
changes slowly) which allows us to write 

+ (1 - p,) tanh In ((B.2) 

with pr the fraction t jt. To proceed further we then make three key assumptions: 

(i) |ut| and |e| are sufficiently small that we can approximate the tanh term by a linear 
function. 


(ii) The right steps (and therefore also the X left steps) are uniformly distributed 
throughout the trajectory. 

(iii) Vr, e and Pr are mutually independent. 

All three of these assumptions are expected to fail as AT approaches 1 but they do 
facilitate analytical progress for AT 3> 1. 

First, we use assumption (|i]) to make a linear expansion 

Vt ^ PrVr + {I - Pr) , (B.3) 

and trivially rearrange to obtain 


Vt 


XT pr 


-Vr + 


1 Pr 


, , (B.4) 

AT - 1 + AT - 1 + V 2 y 

where for convenience we have rescaled to e' which has a standard logistic distribution 
with variance = vr^/S. To obtain the distribution of pr, we first consider separately 
the fractional times pjl and p~ for the last records corresponding to right and left 
steps respectively. Each of the previous steps is equally likely to have produced 
the maximum value so, with assumption (|^, we have p^ ~ Uniform(0,1) in the long¬ 
time limit. Then, by straightforward calculation, p^ = max(py,p“) is governed by a 
triangular distribution on [0,1] with mode 1. 

Now, in the long-time limit, we expect the distribution of Vr to be the same as 
that of Vt, both characterized by standard deviation ay- Hence using the independence 
assumption (iii), together with the symmetry of trajectories around zero, we obtain 

(B.5) 


4 


p(AT)4 + g(AT)^ 


where the functions p{x) and q{x) are given by expectations with respect to the 
distribution of pr- 

2" 


p{x) = E 


q{x) = E 


xpr 


X - l-\- Pr 
1 Pr 

X -1 + Pr 


= X 


—2-\-9x — 6x‘^ -\- 6x{x — 1)^ In 


X 


X — 1 


= 1 — 6a; -I- 2x{3x — 2) In 


X 


X — 1 


(B.6) 


(B,7) 
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TT 


(^V 


q{XT) 


2y/3y 1 -p(AT)' 


(B.8) 


The expression in ( |B.8 ) clearly diverges at AT = 1 bnt we can estimate its range 
of applicability by considering assnmption 0. Specihcally, x approximates tanh(a:) to 


within 10% for |a:| < 0.55 so, since Vt and e are fairly strongly correlated, we reqnire 

a,' 


XT 2\T 


< 0.55 


(B.9) 


which is satished for AT > 2.3. Indeed (B.8) is seen to provide a reasonable 
approximation to the simulation data in this regime (cf. hgurej^. The small remaining 
discrepancy is probably mainly due to the failure of assumption ([^; in particular, Vr 
is not strictly independent of e. 
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